Topological Evolution of Dynamical Networks: 
Global Criticality from Local Dynamics* 
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We evolve network topology of an asymmetrically connected threshold network by a simple local 
rewiring rule: quiet nodes grow links, active nodes lose links. This leads to convergence of the 
average connectivity of the network towards the critical value K c — 2 in the limit of large system 
size N. How this principle could generate self-organization in natural complex systems is discussed 
for two examples: neural networks and regulatory networks in the genome. 
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Networks of many interacting units occur in diverse 
areas as, for example, gene regulation, neural networks, 
food webs in ecology, species relationships in biological 
evolution, economic interactions, and the organization of 
the internet. For studying statistical mechanics prop- 
erties of such complex systems, discrete dynamical net- 
works provide a simple testbed for effects of globally in- 
teracting information transfer in network structures. 

One example is the threshold network with sparse 
asymmetric connections. Networks of this kind were first 
studied as diluted, non-symmetric spin glasses M and di- 
luted, asymmetric neural networks . For the study of 
topological questions in networks, a version with discrete 
connections = ±1 is convenient and will be considered 
here, ft is a subset of Boolean networks ^|!| with sim- 
ilar dynamical properties. Random realizations of these 
networks exhibit complex non-Hamiltonian dynamics in- 
cluding transients and limit cycles ||J^]- In particular, a 
phase transition is observed at a critical average connec- 
tivity K c with lengths of transients and attractors (limit 
cycles) diverging exponentially with system size for an 
average connectivity larger than K c . A theoretical anal- 
ysis is limited by the non-Hamiltonian character of the 
asymmetric interactions, such that standard tools of sta- 
tistical mechanics do not apply (j]]. However, combinato- 
rial as well as numerical methods provide a quite detailed 
picture about their dynamical properties and correspon- 
dence with Boolean Networks PHl^l . 

While basic dynamical properties of interaction net- 
works with fixed architecture have been studied with such 
models, the origin of specific structural properties of net- 
works in natural systems is often unknown. For example, 
the observed average connectivity in a nervous structure 
or in a biological genome is hard to explain in a frame- 
work of networks with a static architecture. For the case 
of regulation networks in the genome, Kauffman postu- 
lated that gene regulatory networks may exhibit proper- 
ties of dynamical networks near criticality P,^6|. How- 
ever, this postulate does not provide a mechanism able to 
generate an average connectivity near the critical point. 
An interesting question is whether connectivity may be 
driven towards a critical point by some dynamical mech- 



anism. In the following we will sketch such an approach 
in a setting of an explicit evolution of the connectivity of 
networks. 

Network models of evolving topology, in general, have 
been studied with respect to critical properties earlier 
in other areas, e.g., in models of macro-evolution [fj"7| . 
Network evolution with a focus on gene regulation has 
been studied first for Boolean networks in |18| observing 
self-organization in network evolution, and for threshold 
networks in |p^| . Combining the evolution of Boolean 
networks with game theoretical interactions is used to 
model networks in economy p(jf . 

In a recent paper Christensen et al. plf introduce a 
static network with evolving topology of undirected links 
that explicitly evolves towards a critical connectivity in 
the largest cluster of the network. In particular they 
observe for a neighborhood-oriented rewiring rule that 
the connectivity of the largest cluster evolves towards 
the critical K c — 2 of a marginally connected network. 
Motivated by this work we here consider the topological 
evolution of threshold networks with asymmetric links 
to study how local rules may affect global connectivity 
of a network, including the entire set of clusters of the 
network. In the remainder of this Letter we define a 
threshold network model with a local, topology-evolving 
ride. Then numerical results are presented that indicate 
an evolution of topology towards a critical connectivity 
in the limit of large system size. Finally, we discuss 
these results with respect to other mechanisms of self- 
organization and point to possible links with interaction 
networks in natural systems. 

Let us consider a network of N randomly intercon- 
nected binary elements with states ai = ±1. For each 
site i, its state at time t + 1 is a function of the inputs it 
receives from other elements at time i: 
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The interaction weights take discrete values Cy = ±1, 
with Cij = if site i does not receive any input from 
element j. In the following, the threshold parameter h 
is set to zero. The dynamics of the network states is 
generated by iterating this rule starting from a random 
initial condition, eventually reaching a periodic attractor 
(limit cycle or fixed point). 

Then we apply the following local rewiring rule to a 
randomly selected node i of the network: 
If node i does not change its state during the at- 
tractor, it receives a new non-zero link Cy from 
a random node j. If it changes its state at least 
once during the attractor, it loses one of its non- 
zero links c^. 

Iterating this process leads to a self-organization of the 
average connectivity of the network. 

To be more specific, let us now describe one of several 
possible realizations of such an algorithm in detail. We 
define the average activity A(i) of a site i 
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where the sum is taken over the dynamical attractor of 
the network defined by T\ and T 2 . For practical purposes, 
if the attractor is not reached after T max updates, A(i) 
is measured over the last T max /2 updates. This avoids 
exponential slowing down by long attractor periods for 
an average connectivity K > 2. The algorithm is then 
defined as follows: 

(1) Choose a random network with an average connec- 
tivity K ini . 

(2) Choose a random initial state vector <r(0) = 
(a 1 (0),...,a N (0)). 

(3) Calculate the new system states <?(£), t = 1,...,T 
according to eqn. (2), using parallel update of the N sites. 

(4) Once a previous state reappears (a dynamical at- 
tractor is reached) or otherwise after T max updates the 
simulation is stopped. Then change the topology of the 
network according to the following local rewiring rule: 

(5) A site i is chosen at random and its average activity 
A(i) is determined. 

(6) If = 1, i receives a new link Cy from a site j 
selected at random, choosing = +1 or —1 with equal 
probability. If \A(i)\ < 1, one of the existing non-zero 
links of site i is set to zero. 

(7) Finally, one non-zero entry of the connectivity- 
matrix is selected at random and its sign reversed. 

(8) Go to step number 2 and iterate. 

The fluctuations introduced in step 7 as random spin re- 
versals are motivated by structurally neutral noise often 
observed in natural systems. Omitting this step docs 
not change the basic behavior of the algorithm, how- 
ever, the distribution of the number of inputs per node 
then evolves away from a Poissonian, thereby increasing 
the fraction of nodes with many inputs. The resulting 



dynamics only differs from the original algorithm in a 
slightly larger connectivity K ev of the evolved networks. 
This effect vanishes ~ 1/N with increasing system size. 

The typical picture arising from the model as defined 
above is shown in Fig. 1 for a system of size N = 1024. 
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FIG. 1. Evolution of the average connectivity of thresh- 
old networks rewired according to the rules described in the 
text, for N = 1024 and two different initial connectivities 
(K ini = 1.5 and Ki ni = 3.0). Independent of the initial con- 
ditions chosen at random the networks evolve to an average 
connectivity K ev = 2.55 ± 0.04. The plot shows the time se- 
ries and the corresponding cumulative means for K ev . The 
evolutionary time t is discrete, each time step representing a 
dynamical run on the evolved topology. Individual runs were 
limited to T ma x = 1000 iterations. 

Independent of the initial connectivity, the system 
evolves towards a statistically stationary state with an 
average connectivity K ev (N = 1024) = 2.55±0.04. With 
varying system size we find that with increasing TV the 
average connectivity converges towards K c (which, for 
threshold ft = as considered here, is found at K c = 2), 
see Fig. 2. 
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FIG. 2. The average connectivity of the evolved net- 
works converges towards K c with a scaling law ~ N~ s , 
S — 0.47 ± 0.01. For systems with N < 256 the average 
was taken over 4 ■ 10 6 time steps, for N = 512 and TV = 1024 
over 5- 10° and 2.5-10 time steps, respectively. Finite size ef- 
fects from Tmax = 1000 may overestimate K ev for the largest 
network shown here. 



One observes the scaling relationship 

K ev (N)-2 = c-N- 5 



(4) 



with c = 12.4±0.5 and S = 0.47±0.01. Thus, in the large 
system size limit N — > oo the networks evolve towards the 
critical connectivity K c = 2. 

The self-organization towards criticality observed in 
this model is different from currently known mechanisms 
exhibiting the amazingly general phenomenon of self- 
organized criticality (SOC) p2p^Jl7| . Our model intro- 
duces a (new, and interestingly different) type of mecha- 
nism by which a system self-organizes towards criticality, 
here K — ► K c . This class of mechanisms lifts the notions 
of SOC to a new level. In particular, it exhibits consid- 
erable robustness against noise in the system. The main 
mechanism here is based on a topological phase transition 
in dynamical networks. To see this consider the statisti- 
cal properties of the average activity A(i) of a site i for a 
random network. It is closely related to the frozen com- 
ponent C(K) of the network, defined as the fraction of 
nodes that do not change their state along the attractor. 
The average activity A(i) of a frozen site i thus obeys 
\A(i)\ = 1. In the limit of large N, C(K) undergoes a 
transition at K c vanishing for larger K . With respect to 
the average activity of a node, C{K) equals the proba- 
bility that a random site i in the network has = 1. 
Note that this is the quantity which is checked stochas- 
tically by the local update rule in the above algorithm. 
The frozen component C (K, N) is shown for random net- 
works of two different system sizes N in Fig. 3. 




FIG. 3. The frozen component C(K, N) of random thresh- 
old networks, as a function of the networks' average connec- 
tivities K. For both system sizes shown here (N = 256 and 
N — 1024) the data were measured along the dynamical at- 
tractor reached by the system, averaged over 1000 random 
topologies for each value of K. One observes a transition 
around a value K = Ko approaching K c — 2 for large N. 
A sigmoid function fit is also shown. To avoid trapping in 
exponential divergence of attractor periods for K > 2 the 
simulations have been limited to T max = 10000. The mis- 
match of data and fit for TV = 1024, K > 2.75 is due to this 
numerical limitation. 

One finds that C (K, N) can be approximated by 

C(K, N) = 1{1 + tanh [-a(N) ■ (K - K (N) )]}. (5) 

This describes the transition of C (K, N) at an average 
connectivity K (N) which depends only on the system 
size N. 
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FIG. 4. The finite size scaling of the transition value Ko, 
obtained from sigmoidal fits as shown in FIG. 3. Kq ap- 
proaches K c = 2 with a scaling law ~ JV - ' 3 , j3 = 0.34 ± 0.01. 
The inset shows the scaling behavior of the parameter a(N); 
one finds a(N) ~ TV 7 , 7 = 0.41 ± 0.014. 

One finds for the finite size scaling of Kq(N) that 

K a (N)-2 = a-N-P (6) 

with a = 3.30 ± 0.17 and (3 = 0.34 ± 0.01 (see Fig. 4), 
whereas the parameter a scales with system size as 



i(N) = b ■ N 1 



(7) 



with b = 0.14 ± 0.016 and 7 = 0.41 ± 0.01. Thus we see 
that in the thermodynamic limit N — > 00 the transition 
from the frozen to the chaotic phase becomes a sharp step 
function at -Ko(oo) = K c . These considerations apply 
well to the evolving networks in the rewiring algorithm. 

In addition to the rewiring algorithm as described in 
this Letter, we tested a number of different versions of the 
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model. Including the transient in the measurement of the 
average activity A(i) results in a similar overall behavior 
(where we allowed a few time steps for the transient to de- 
couple from initial conditions) . Another version succeeds 
using the correlation between two sites instead of A(i) 
as a mutation criterion (this rule could be called "anti- 
Hebbian" as in the context of neural network learning) . 
In addition, this version was further changed allowing 
different locations of mutated links, both, between the 
tested sites or just at one of the nodes. Some of these 
versions will be discussed in detail in a separate article 
p4j . All these different realizations exhibit the same ba- 
sic behavior as found for the model above. Thus, the 
mechanism proposed in this Letter exhibits considerable 
robustness. 

An interesting question is whether a comparable mech- 
anism may occur in natural complex systems, in partic- 
ular, whether it could lead to observable consequences 
that cannot be explained otherwise. 

One example where such mechanisms could occur is 
the regulation of connectivity density in neural systems. 
Activity-dependent attachment of synapses to a neuron 
is well known experimentally, for example in the form 
of the gating of synaptic changes by activity correlation 
between neurons |25|]. Such local attachment rules could 
provide a sufficient basis for a collective organization to 
occur as described in this Letter. For symmetric neural 
networks similar rules have been discussed, e.g., in the 
context of "Hebbian unlearning" suppressing spurious 
memories [^6|. In the here studied asymmetric networks, 
however, such rules appear to generate a completely new 
form of self-organization dynamics. As a consequence, an 
emerging average connectivity K ev could be stabilized to 
a specific value mostly determined by local properties of 
the dynamical elements of the system. It would be inter- 
esting to discuss whether synaptic density in biological 
systems could be regulated by such mechanisms. 

Another biological observable of interest is the con- 
nectivity of gene-gene interactions in the expression of 
the genome as first studied by Kauffman M. Whether 
this observable results from any such mechanism clearly 
is an open question. However, one may discuss whether 
biological evolution exerts selection pressure on the sin- 
gle gene level, that results in a selection rule similar to 
our algorithm; E.g., for a frozen regulation gene which is 
practically non-functional to obtain a new function (ob- 
tain a new link), as well as for a quite active gene to 
reduce functionality (remove a link). First experimental 
estimates for the global observable of genome connectiv- 
ity are available for E. coli with a value in the range 2 — 3 
p7| . While it is clearly too early to speculate about the 
mechanisms of global genome organization, it is inter- 
esting to note that the robust self-organizing algorithm 
presented here provides a mechanism that in principle 
predicts a value in this range. 

To summarize, we study topological evolution of asym- 



metric dynamical networks on the basis of a local rewiring 
rule. We observe a network evolution with the average 
connectivity K of the network evolving towards the crit- 
ical connectivity K c without tuning. In the limit of large 
system size N this limit becomes accurate. It is well con- 
ceivable that this form of global evolution of a network 
structure towards criticality might be found in natural 
complex systems. 
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